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ABSTRACT 

This report summarizes the 2006-2007 evaiuation that was focused toward investigating 
one primary question: What changes in tooi-based, student-centered teaching happen as a resuit 
of the infusion of technoiogy and professionai deveiopment? The research methodoiogy invoived 
the use of trained externai researchers from Fiorida EETT schoois conducting muiti-ciass and 
targeted ciassroom observations in each participating schooi during two time periods: baseiine (faii 
2006) and end of year one (spring 2007). A totai of 381 hours of direct ciassroom observations 
were conducted in 845 FL EETT ciassrooms in 41 schoois representing 1 1 districts. Observation 
data were coiiected with the Schooi Observation Measure (SOM®) and the Survey of Computer 
Use (SCU*^. The SOM was used to coiiect data regarding overaii ciassroom activities and the 
SCU was used to assess student use of computers. Both descriptive and inferentiai anaiyses 
were conducted. The Mantei-Fiaentzei procedure was used to infer statisticai differences between 
the faii and spring ciassroom observations. 

Both the SOM and SCU Muiti-Ciass and Targeted observations reveaied significant faii to 
spring increases in the use of teacher-centered practices. For the SOM, significant increases 
were found for both the Muiti-Ciass and Targeted observations for student engagement in “Project- 
based iearning”, “independent inquiry/research on the part of students”, and student use of 
“Technoiogy as a iearning tooi or resource.” The SCU resuits from both the Muiti-Ciass and 
Targeted observations yieided significant increases in students’ overaii use of newer and more up- 
to-date computers (iaptops) and positive trends toward increased uses of production toois and 
internet/research toois to support iearning. A key finding that emerged from the resuits was the 
significant increase in the frequency with which teachers impiemented meaningfui computer 
activities that engaged students in higher-order thinking and probiem soiving through effective use 
of iaptop-based technoiogy toois. 

These first year resuits show promising trends in that the Fiorida EETT program seems to 
be serving as a cataiyst for positive changes from traditionai teaching environments to ones that 
are student-centered and engage iearners in meaningfui use of computers to enhance iearning. 
Fiowever, the data aiso reveai room for continued growth due to the modest frequency with which 
most of these changed practices occurred. An additionai consideration when reviewing the 
evaiuation resuits is the possibie bias that may occur due to observer invoivement in the Fiorida 
EETT program. 
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2006-2007 EVALUATION REPORT: CLASSROOM PRACTICES 



This report summarizes the 2006-2007 ciassroom practices evaiuation resuits of the 
Fiorida EETT program. The overaii purpose of the evaiuation was twofoid: (a) to provide 
evidence of EETT program impiementation progress as demonstrated through ciassroom 
practices and (b) to provide formative evaiuation data of ciassroom practices as a basis for guiding 
improvement pianning. The evaiuation question, participants, instrumentation, procedures, and 
resuits are provided in the sections to foiiow. 

EVALUATION QUESTION 

This evaiuation was focused toward investigating one primary question: What changes in 
tooi-based, student-centered teaching happen as a resuit of the infusion of technoiogy and 
professionai deveiopment? 

METHODOLOGY 

The methodoiogy chosen to address the evaiuation question was to conduct direct 
ciassroom observations in each participating Fiorida EETT schooi during two time periods: 
baseiine (faii 2006) and end of year one (spring 2007). Trained externai researchers from Fiorida 
EETT schoois conducted both muiti-ciass and targeted observations. The two types of 
observations were used to more fuiiy investigate program impact on ciassroom practices. The 
intent of the muiti-ciass observations was to identify iaptop integration practices that routineiy 
occur on a day-to-day basis. Converseiy, targeted observations were prescheduied and aiiowed 
teachers to demonstrate their best practices with regard to integrating the use of iaptops into 
ciassroom instruction. Detaiis of the participants, observation measures, and procedures are 
provided beiow. 

Participants 

A totai of 428 Fiorida iaptop teachers and approximateiy 8,500 students from 41 schoois 
participated in the observation activities. The schoois represented 1 1 districts from rurai Fiorida as 
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well as major metropolitan areas distributed across the state. Of the 41 schools, 12 were 
elementary schools, 17 middle schools, and 12 were high schools. 

Measures and Procedures 

External researchers completed extensive training to conduct both multi-class and 
targeted classroom observations of Florida EETT classrooms with two data collection Instruments: 
the School Observation Measure (SOM®), and the Survey of Computer Use (SCU®). The SOM 
was used to collect data regarding overall classroom activities and the SCU was used to assess 
student use of computers. 

The multi-class procedure Involved an observer visiting 10-12 randomly selected laptop 
classrooms for 15 minutes each during a three-hour visitation period. At the conclusion of the 
three-hour visit, the observer summarized the frequency with which the SOM and SCU strategies 
were observed across all classes on a data summary form. Targeted observations Involved 
observing laptop classrooms during prearranged 45- to 90-mlnute sessions In which randomly 
selected Florida EETT teachers were asked to Implement a prepared lesson that Integrated the 
use of laptops. Notes forms were completed every 15 minutes of the lesson and then summarized 
on a data summary form. 

SOM. The SOM was developed to determine the extent to which different common and 
alternative teaching practices are used throughout an entire school or program (Ross, Smith, & 
Alberg, 1999). The observer examines classroom events and activities descriptively, not 
judgmentally. Notes are taken relative to the use or nonuse of 24 target strategies. The notes 
form also contains two global Items that use a three-point scale (low, moderate, high) to rate, 
respectively, the degree of academically focused Instructional time and degree of student attention 
and Interest. The frequency Is recorded via a 5-polnt rubric that ranges from (0) Not observed to 
(4) Extensively. The same 5-polnt scale Is used to summarize how frequently high academically 
focused class time and high student Interest/attention are observed. 

To ensure the reliability of data, observers receive a manual providing definitions of terms, 
examples and explanations of the target strategies, and a description of procedures for completing 
the Instrument. The target strategies Include traditional practices (e.g., direct Instruction and 
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independent seatwork) and alternative, predominately student-centered methods associated with 
educational reforms (e.g., cooperative learning, project-based learning, inquiry, discussion, using 
technology as a learning tool). The strategies were identified through surveys and discussions 
involving policy makers, researchers, administrators, and teachers, as those most useful in 
providing indicators of schools’ instructional philosophies and implementations of commonly used 
reform designs (Ross, Smith, Alberg, & Lowther, 2001). 

In a 2004 reliability study reported by Sterbinsky, Ross and Burk, observer ratings were 
within one category for 96% of the multi-class observations and for 91% of the targeted 
observations. 

SCU. A companion instrument to the SOM is the Survey of Computer Use (SCU) 

(Lowther & Ross, 2001). The SCU was completed as part of the SOM observation sessions, 
during which SCU data were also recorded in 15-minute intervals and then summarized on an 
overall data form. 

The SCU was designed to capture exclusively student access to, ability with, and use of 
computers rather than teacher use of technology by recording four types of data: (a) computer 
capacity and currency, (b) configuration, (c) student computer ability and (d) student activities 
while using computers. Computer capacity and currency is defined as the age and type of 
computers available for student use and whether or not Internet access was available. 
Configuration refers to the number of students working at each computer (e.g., alone, in pairs, in 
small groups). Student computer ability was assessed by recording the number of students who 
were computer literate (i.e., easily used software features/menus) and the number of students who 
easily used the keyboard. 

The next section of the SCU focuses on student use of computers with regard to: the 
types of activities, the subject areas of activities, and the software being used. The computer 
activities are divided into four categories based on the type of software tool: production tools. 
Internet/research tools, educational software, and testing software. Within each category, primary 
types of software are identified. For example, under Production Tools, the software includes: word 
processing, databases, spreadsheets, draw/paint/graphics, presentation (e.g., PowerPoint®), 



Florida’s EETT 2006-2007 Evaluation Report 4 



authoring (e.g., KidPix®), concept mapping (e.g., inspiration), and pianning (MS Project®). For 
the internet/Research Toois, three types of software are inciuded: internet browser, CD reference 
materiais, and communications (e.g., emaii, iistservs, and chat rooms). The Educationai Software 
category aiso has three types of software: driii/practice/tutoriai, probiem-soiving (e.g., 
Riverdeep™), and process toois (e.g.. Author’s Tooikit™). Testing Software has 
individuaiized/tracked (Acceierated Reader™) and generic types. With this type of recording 
system, severai activities can be noted during the observation of one student working on a 
computer. For exampie, if a student gathered data from the internet, created a graph from the 
data, and then imported the graph into a PowerPoint presentation, the observer wouid record three 
types of software toois as being observed: internet browser, spreadsheet, and presentation. 

This section of the SCU ends by identifying the subject area of each computer activity. 

The categories inciude: ianguage arts, mathematics, science, sociai studies, other, and none. The 
computer activities and software being used are summarized and recorded using a five-point rubric 
that ranges from (0) Not Observed to (4) Extensiveiy observed. The finai section of the SCU is an 
“Overaii Rubric” designed to assess the degree to which the activity refiects “meaningfui use” of 
computers as a toolto enhance iearning. The rubric has four ieveis: 1) Low-ievei use of 
computers, 2) Somewhat meaningfui, 3) Meaningfui, and 4) Very meaningfui. Reiiabiiity data for 
the SCU (Sterbinsky & Burke, 2004) show that observer ratings were within one category for 97% 
of the muiti-ciass observations and for 91% of the targeted observations. 

Data Collection 

A data coiiection summary of the Fiorida EETT ciassroom observations is presented in 
Tabie 1 . A totai of 381 hours of direct ciassroom observations were conducted in 845 FL EETT 
ciassrooms. 



Florida’s EETT 2006-2007 Evaluation Report 5 




TABLE 1 



Data Collection Summary 







Number 


Classrooms 


Schools 








Collected 


Observed 


Involved 




Type 


Instrument 


Fall 


Spring 


Fall 


Spring 


Fall 


Spring 


Procedure 


Multi-Class 

Observations 


SOM 

SCU 


34 

35 


54 

54 


284 


447 


41 

41 


41 

41 


Multi-class observations were 
three- hour sessions in which 
external researchers observed 
about 1 0 randomly selected 
classes for 1 5 minutes each. 
The purpose was to obtain a 
program-wide perspective on 
common teaching practices and 
the use of technology in EETT 
laptop classrooms. 


Targeted 


SOM 


38 


76 


38 


76 


41 


41 


Targeted observations were pre- 


Classroom 


SCU 


37 


70 






41 


41 


arranged one-hour sessions in 


Observations 
















which EETT teachers were 
asked to demonstrate a 


















prepared lesson using laptops. 
Note forms were completed 
every 15 minutes of the lesson. 



Data Analysis 

The majority of observation results for both SOM and SCU are in an ordinal scale of 
measurement, which usually fails to have a normal distribution. In addition, the observations in 
school year 2006-07 were collected twice: once in fall 2006 (pre), then in spring 2007 (post). Thus, 
to account for data stratified in nature and with particular characteristics (i.e., ordinal response 
data and repeated measures), the Mantel-Haentzel procedure was used to infer statistical 
differences between the pre- and post-classroom observations. 

Two statistics, Qs/wwand Qcsmh, were reported. The statistic Osmh was used to measure 
the trend (e.g., increase or decrease) in the value of responses between observations, while 
Qcsmh was used to detect whether the mean responses were the same across the measurement 
time points (pre = Fall and post = Spring). As data from both SOM and SCU are complete (i.e., 
without missing values), the Qsmh and Qcsmh outcomes are identical in value (see Tables 2, 3 and 
4). For multiple comparisons, the Bonferroni adjustment was used on the alpha level to control the 
experimental-wise error. However, because of the strictness of the Bonferroni adjustment, 
observations that approached the adjusted significance level were also reported. Effect sizes were 
computed by dividing the mean difference by the pooled standard deviation. Except where noted. 
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a positive sign before the effect size is indicative of outcomes favoring the spring (post) over the 
faii (pre) observation resuits, while a negative sign reveals that the fall had higher ratings than the 
spring. 

RESULTS 

The results of the study are presented below by data collection strategy: multi-class and 
targeted observations. Within these categories, data are presented by observation measure 
(SOM; SCU). In the Conclusion section, findings are synthesized across the two instruments to 
address the evaluation question. 

Multi-Class Observation Results 

A total of 89 multi-class observations (35 Fall and 54 Spring) were conducted in 41 Florida 
schools, which yielded approximately 267 hours of direct observation. The SOM and SCU 
instruments were used to collect data from unannounced, random visits to 284 classrooms in the 
fall and 447 classrooms in the spring. Results from the multi-class visits are presented below by 
observation instrument. 

Multi-Class SOM 

The SOM results from the fall and spring observations revealed changes in teaching 
strategies as well as student activities (see Table 2). Overall, the greatest fall (baseline) to spring 
differences were seen in increased “Fligh student attention, interest, and engagement” (Fall M = 
2.18, Spring M = 2.91, ES = -i-l .00) and a decrease in the use of traditional “Independent 
seatwork” (Fall M = 2.68, Spring M = 1 .69, ES = -1 .00). Other notable differences include greater 
use of “Project-based learning” (ES = + 0.93), “Teacher acting as coach/facilitator” (ES = + 0.78), 
“Cooperative/Collaborative learning” (ES= + 0.62), “Independent inquiry/research” (ES= + 0.63), 
and “High academically focused class time” (ES= + 0.61), with an understandable decline in the 
use of “Direct instruction” (Fall M = 2.91 , Spring M = 2.1 9, ES= -0.82). Of particular interest are the 
positive changes in computer use. While use of “Computers as a delivery tool” (a teacher-centered 



Florida’s EETT 2006-2007 Evaluation Report 7 




activity) showed a decrease (ES= -.40), student use of “Technoiogy as a learning tool” (student- 
centered) showed a impressive increase. (ES=-i-.61) 

TABLE 2 

Multi Class School Observation Measure (SOM) 

Fall (Baseline) N=34 (284 Classrooms) 

Spring N=54 (447 Classrooms) 







Percent Observed 


Florida EETT 


National Norm 


The extent to which each of the following was 


None or 




Frequently 

or 




Standard 






Standard 


observed in the classroom. 




Rarely 


Occasionally 


Extensively 


Mean 


Deviation 


ES 


Mean 


Deviation 


Instructional Orientation 




















Direct instruction (lecture) 


Baseline 


5.9 


20.6 


73.5 


2.91 


0.83 


-0.82 


2.77 


1.01 


Spring 


22.3 


38.9 


38.9 


2.19 


0.87 




Baseline 


94.1 


5.9 


0.0 


0.38 


0.60 








Team teaching 


Spring 


88.9 


11.1 


0.0 


0.52 


0.69 


+0.15 


0.81 


0.94 




Baseline 


73.6 


26.5 


0.0 


0.85 


0.82 


+0.62 


1.08 


0.98 


Cooperative/collaborative learning 


Spring 


48.2 


33.3 


18.5 


1.46 


1.06 


Individual tutoring (teacher, peer, aide. 


Baseline 


82.3 


14.7 


2.9 


0.62 


0.85 


-0.25 


0.77 


0.94 


adult volunteer) 


Spring 


90.7 


9.3 


0.0 


0.44 


0.66 


Classroom Organization 


Baseline 


85.3 


5.9 


8.8 


0.71 


1.12 


0.00 


1.16 


1.33 


Ability groups 


Spring 


81.5 


5.6 


13.0 


0.72 


1.16 




Baseline 


94.1 


5.9 


0.0 


0.21 


0.54 








Multi-age grouping 


Spring 


87.1 


9.3 


3.7 


0.39 


0.81 


+0.30 


0.58 


1.03 




Baseline 


94.2 


5.9 


0.0 


0.24 


0.55 








Work centers (for individuals or groups) 


Spring 


94.4 


5.6 


0.0 


0.33 


0.58 


+0.17 


1.35 


1.14 


Instructional Strategies 

Higher level instructional feedback (written 


Baseline 


55.9 


23.5 


20.5 


1.35 


1.20 


+0.17 


1.44 


1.15 


or verbal) to enhance student learning 


Spring 


53.7 


20.4 


26.0 


1.61 


1.16 


Integration of subject areas 


Baseline 


91.2 


5.9 


2.9 


0.59 


0.86 


+0.20 


0.54 


0.80 


(interdisciplinary/thematic units) 


Spring 


79.7 


7.4 


13.0 


0.80 


1.11 


Project-based learning 


Baseline 


79.4 


20.6 


0.0 


0.59 


0.82 










Spring 


50.0 


18.5 


31.5 


1.56 


1.33 


+0.93 


0.45 


0.76 


Use of higher-level questioning strategies 


Baseline 


61.8 


20.6 


17.6 


1.26 


1.08 










Spring 


53.7 


16.7 


29.7 


1.59 


1.32 


+0.25 


1.63 


1.12 


Teacher acting as a coach/facilitator 


Baseline 


58.8 


20.6 


20.5 


1.26 


1.24 










Spring 


29.7 


35.2 


35.2 


2.19 


1.07 


+0.78 


2.29 


1.18 


Parent/community involvement in learning 


Baseline 


100.0 


0.0 


0.0 


0.06 


0.24 


0.00 


0.31 


0.60 


activities 


Spring 


98.2 


0.0 


1.9 


0.13 


0.58 



Student Activities 



Independent seatwork (self-paced 
worksheets, individual assignments) 


Baseline 

Spring 


12.7 

46.3 


20.6 

31.5 


64.7 

22.2 


2.68 

1.69 


1.04 

1.02 


-1.00 


2.41 


0.98 


Experiential, hands-on learning 


Baseline 

Spring 


73.5 

59.2 


23.5 

18.5 


2.9 

22.3 


0.85 

1.35 


0.89 

1.26 


+0.45 


1.20 


1.00 
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Percent Observed 



National Norm 



Florida EETT 



The extent to which each of the following was 


None or 




Frequently 

or 




Standard 






Standard 


observed in the classroom. 




Rarely 


Occasionally Extensively 


Mean 


Deviation 


ES 


Mean 


Deviation 


Systematic individual instruction 


Baseline 


97.1 


2.9 


0.0 


0.32 


0.53 








(differentiated assignments geared to 
individual needs) 


Spring 


90.8 


5.6 


3.7 


0.39 


0.9 


-hO.14 


0.44 


0.76 


Sustained writing/composition (self- 
selected or teacher-generated topics) 


Baseline 

Spring 


91.2 

85.2 


5.9 

13.0 


2.9 

1.9 


0.47 

0.65 


0.75 

0.78 


+0.13 


0.75 


0.86 




Baseline 


88.2 


5.9 


5.9 


0.53 


0.86 








Sustained reading 


Spring 


85.1 


14.8 


0.0 


0.70 


0.72 


+0.25 


1.08 


0.98 


Independent inquiry/research on the part 
of students 


Baseline 

Spring 


85.3 

57.4 


8.8 

25.9 


5.9 

16.7 


0.76 

1.43 


0.85 

1.02 


+0.63 


0.32 


0.65 


Student discussion 


Baseline 

Spring 


67.7 

59.2 


32.4 

33.3 


0.0 

7.5 


1.00 

1.09 


0.82 

1.07 


+0.10 


0.89 


1.11 


Technology Use 




















Computer for instructional delivery (e.g. 
CAI, drill & practice) 


Baseline 

Spring 


20.5 

38.9 


55.9 

35.2 


23.5 

25.9 


2.09 

1.72 


0.90 

1.12 


-0.40 


0.96 


0.97 


Technology as a learning tool or resource 


Baseline 


58.9 


26.5 


14.7 


1.35 


1.15 








(e.g. Internet research, spreadsheet or 
database creation) 


Spring 


27.8 


35.2 


37.0 


2.15 


1.07 


+0.61 


0.80 


0.98 


Assessment 


Baseline 


94.2 


0.0 


5.9 


0.29 


0.76 








Performance assessment strategies 


Spring 


87.1 


7.4 


5.6 


0.52 


0.86 


+0.23 


0.50 


0.83 


Student self-assessment (portfolios, 
individual record books) 


Baseline 

Spring 


91.1 

98.2 


5.9 

1.9 


2.9 

0.0 


0.24 

0.24 


0.70 

0.47 


0.00 


0.32 


0.65 


Summary Items 




















High academically focused class time 


Baseline 


5.9 


38.2 


55.9 


2.59 


0.74 


+0.61 


3.33 


0.80 


Spring 


0.0 


18.5 


81.5 


3.04 


0.64 


High level of student attention, interest, 


Baseline 


14.7 


55.9 


29.4 


2.18 


0.72 


+1.00 


3.12 


0.83 


engagement 


Spring 


1.9 


25.9 


72.3 


2.91 


0.73 



Scale: 0 = Not Observed; 1 = Rarely; 2 = Occasionally; 3 = Frequently; 4 = Extensively 



SOM Multi-Class Inferential Analyses 

As previously mentioned, there are a total of 26 items on the SOM, which are evaluated 
using an ordinal 5-point Likert scale that ranges from “0 = Not observed” to “4 = Extensively”. The 
SOM multi-class inferential analysis outcomes are presented in Table 3 and Figure 1. The FL 
EETT classrooms that were observed had significant increases on four SOM items: (1) “Project- 
based learning” {p<.001), (2) “Independent inquiry/research on the part of students” (p=.001), (3) 
“Technology as a learning tool or resource” (p<.001), and (4) “High level of student 
attention/interest/engagement” (p<.001), and significant decreases in “Direct instruction” {p=.001) 
and “Independent seatwork” (p<.001). In addition, two items, (1) “Teacher acting as a 
coach/facilitator” (p=.0021) and (2) “High academically focused class time” (p=.0021), approached 
significance with large associated effect sizes. 
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TABLE 3 

SOM Multi-Class Means Comparison between Fall and Spring Using Mantel-Haentzel Test 



Item 


Qsmh 


P 


Qcsmh 


P 


Instructional Orientation 










Direct instructicn (lecture) 


10.639* 


.001 


10.639* 


.001 


Team teaching 


0.519 


.471 


0.519 


.471 


Cooperative/ccllaborative learning 


6.721 


.010 


6.721 


.010 


Individual tutoring (teacher, peer, aide, adult volunteer) 

Classroom Organization 


2.045 


.153 


2.045 


.153 


Ability groups 


0.288 


.592 


0.288 


.592 


Multi-age grouping 


2.253 


.133 


2.253 


.133 


Work centers (for individuals or groups) 

Instructional Strategies 


0.828 


.363 


0.828 


.363 


Higher-level instructional feedback (written or verbal) to enhance 
student learning 


0.865 


.352 


0.865 


.352 


Integration of subject areas (interdisciplinary/thematic units) 


2.124 


.145 


2.124 


.145 


Preject-based learning 


13.627* 


<.001 


13.627* 


<.001 


Use of higher-level questioning strategies 


2.139 


.144 


2.139 


.144 


Teacher acting as a coach/facilitator 


9.459 


.0021 


9.459 


.0021 


Parent/cemmunity involvement in learning activities 

Student Activities 


0.441 


.507 


0.441 


.507 


Independent seatwork (self-paced worksheets, individual 
assignments) 


13.133* 


<.001 


13.133* 


<.001 


Experiential, hands-on learning 


4.573 


.033 


4.573 


.033 


Systematic individual instruction (differential assignments geared tc 
individual needs) 


0.180 


.671 


0.180 


.671 


Sustained writing/composition (self-selected or teacher-generated 
tcpics) 


0.636 


.425 


0.636 


.425 


Sustained reading 


0.656 


.418 


0.656 


.418 


Independent inquiry/research on the part of students 


10.565* 


.001 


10.565* 


.001 


Student discussion 


1.323 


.250 


1.323 


.250 


Technology Use 










Computer fcr instructional delivery (e.g., CAI, drill & practice) 


2.760 


.097 


2.760 


.097 


Techncicgy as a learning toel or resource (e.g., Internet research, 
spreadsheet creation, multi-media, CD Rem, Laser disk) 


10.419* 


.001 


10.419* 


.001 


Assessment 










Performance assessment strategies 


0.590 


.442 


0.590 


.442 


Student self-assessment (pertfolios, individual record books) 

Summary Items 


0.001 


.974 


0.001 


.974 


High academically focused class time 


9.486 


.0021 


9.486 


.0021 


High level of student attentien/interest/engagement 


16.988* 


<.001 


16.988* 


<.001 



'statistically significant at alpha<.0019 
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Figure 1. Multi-Class SOM: Significant Fall vs. Spring Differences 



Multi-Class SCU 

As seen in Table 4, the number of classrooms with “1 1 or more” computers available for 
student use increased from 57.1% in the fall to 72.2% in the spring, with 98.1% of the computers 
observed in the spring considered as “Up-to-date”. There was also an increase (Fall = 28.6%; 
Spring = 51 .9%) in the percentage of classrooms in which the laptops were used by “nearly all” of 
the students, as well as an increase in the percentage of students rated with “very good” computer 
literacy skills (Fall = 31.4%; Spring = 46.3%) and keyboarding skills (Fall = 14.3%; Spring = 
35.2%). As would be expected, there was a decrease in the percentage of classrooms in which 
desktop computers were “Frequently” to “Extensively” observed (Fall = 17.2%; Spring = 8.6%), 
while laptop availability increased (Fall = 37.2%; Spring = 74.1%). 

Students were observed using a variety of computer applications during the multi-class 
visits, with notable increased usage of three key tools from fall to spring. Specifically, the greatest 
increase was seen in student use of “Internet Browsers” (Fall M= 1 .23, Spring M = 2.17; ES = 
-1-0.80). Students also more frequently used “Draw, paint, and/or graphics” software (ES= -t-0.70) 
and “Presentation” software (ES = -t-0.58). The most frequently observed subject area of the 
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computer activities was language arts, which was seen during 74.1% of the “Production tool” use 
and 59.3% of the “Internet/Research tool” use. 

Meaningfulness of Computer Activities. The data revealed very positive trends. The largest 
gain was seen in the category “Meaningful use of computers” (Fall M = 0.94, Spring M = 1 .87, ES 
= +0.83), which is defined as, “activities were problem-based, required some critical thinking skills, 
and some use of computer applications to locate and/or process information or some manipulation 
of educational software variables to reach solutions.” Additionally it should be noted that this 
category was observed Extensively to Occasionally in 59.3% of the spring multi-class visits. “Very 
meaningful use of computers” also had a substantial gain (Fall M = 0.31 , Spring M = 1 .1 1 , ES = 
+0.77). Conversely, a large drop was seen in “Low level Use of Computers” (Fall M = 1 .1 7, Spring 
M = 0.78, ES = -0.36), defined as “activities in general required no critical thinking, e.g., used 
computer applications for copying text or free-time drawing, or used educational software for drill & 
practice, tutorials, or games”, as it was only seen “Frequently” to “Extensively” in 5.6% of the 
spring visits. 
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TABLE 4 

Multi-Class SCU Data Summary 

Fall (Baseline) N = 35 

Spring N = 54 



Computer Configuration 


Florida 

EETT 


Percent 

Observed 


Percentages of classrooms with the following numbers of computers or digital tools: 




None, one, or 2 -4 


Baseline 

Spring 


31.4 

7.4 


5-10 


Baseline 

Spring 


11.2 

20.4 


1 1 or more 


Baseline 

Spring 


57.1 

72.2 


Percentages of classrooms in which the majority 


of computers were: 




Up-to-date 


Baseline 

Spring 


74.3 

98.1 


Aging, but adequate 


Baseline 

Spring 


25.7 

0.0 


Outdated/limited capacity 


Baseline 

Spring 


0.0 

1.9 


Percentages of classrooms in which the majority 


of computers were: 




Connected to the Internet 


Baseline 

Spring 


91.4 

92.6 


Student Computer Use 


Percentage of classrooms in which computers or digital tools were used by: 




Few (less than 10%) to Some (about 10- 
50%) students 


Baseline 

Spring 


28.6 

13.0 


Most (about 51-90%) students 


Baseline 

Spring 


34.3 

31.5 


Nearly all (91-100%) students 


Baseline 

Spring 


28.6 

51.9 


Percentage of classrooms in which students worked with computers or digital tools: 




Alone 


Baseline 

Spring 


88.6 

75.9 


In pairs or small groups 


Baseline 

Spring 


2.9 

18.5 


Percentage of classrooms in which student computer literacy skills were: 




Poor 


Baseline 

Spring 


5.7 

3.7 


Moderate 


Baseline 

Spring 


40.0 

31.5 


Very good 


Baseline 

Spring 


31.4 

46.3 


Not observed 


Baseline 

Spring 


22.9 

18.5 


Percentage of classrooms in which student keyboarding skilis were: 




Poor 


Baseline 

Spring 


8.6 

1.9 


Moderate 


Baseline 

Spring 


40.0 

38.9 


Very good 


Baseline 

Spring 


14.3 

35.2 


Not observed 


Baseline 

Spring 


37.1 

24.1 
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Table 4 Continued 



Percent Observed 


Digital Devices Used by Students 




Not or Rarely Observed 


Occasionally 


Frequently or 
Extensively 


Desktop Computers 


Baseline 

Spring 


77.1 

74.0 




14.3 

18.5 


17.2 

8.6 


Laptop Computers 


Baseline 

Spring 


37.1 

9.3 




25.7 

16.7 


37.2 

74.1 


Personal Data Assistants (PDA) 


Baseline 

Spring 


100.0 

98.1 




0.0 

1.9 


0.0 

0.0 


Graphing Calculator 


Baseline 

Spring 


100.0 

96.3 




0.0 

3.7 


0.0 

0.0 


Information Processor (e.g., Alphaboard) 


Baseline 

Spring 


100.0 

100.0 




0.0 

0.0 


0.0 

0.0 


Digital Accessories (e.g., camera, scanner, 


Baseline 


94.3 




5.7 


0.0 


probes) 


Spring 


77.8 




11.1 


11.2 



Note. Item percentages may not total 100% because of missing data. 



Student Computer Activities 

The extent to which each of the following was 
observed in the classroom. 


Percent Observed 


Florida EETT 


National Norm 


None Frequently 

or or 

Rarely Occasionally Extensively 


Standard 

Mean Deviation E$ 


Standard 
Mean Deviation 


Production Tools Used by Students 





Baseline 


71.5 


17.1 


11.4 


0.97 


1.20 


+0.32 


0.50 


0.90 


Word Processing 


Spring 


55.5 


18.5 


25.9 


1.44 


1.33 




Baseline 


100.0 


0.0 


0.0 


0.00 


0.00 








Database 


Spring 


98.1 


1.9 


0.0 


0.07 


0.33 


+0.47 


0.02 


0.19 




Baseline 


97.2 


2.9 


0.0 


0.14 


0.43 


+0.35 


0.07 


0.35 


Spreadsheet 


Spring 


88.9 


11.1 


0.0 


0.30 


0.66 




Baseline 


94.3 


5.7 


0.0 


0.23 


0.55 








Draw/Paint/Graphics 


Spring 


74.1 


14.8 


11.2 


0.80 


1.12 


+0.70 


0.19 


0.57 




Baseline 


77.2 


17.1 


5.8 


0.77 


1.03 








Presentation (e.g., MS PowerPoint) 


Spring 


55.5 


13.0 


31.5 


1.50 


1.37 


+0.58 


0.26 


0.68 




Baseline 


94.3 


2.9 


2.9 


0.17 


0.75 


+0.25 


0.02 


0.20 


Authoring (e.g., HyperStudio) 


Spring 


83.3 


11.1 


5.6 


0.43 


0.90 




Baseline 


100.0 


0.0 


0.0 


0.20 


0.41 


+0.32 


0.08 


0.40 


Concept Mapping (e.g., Inspiration) 


Spring 


88.9 


9.3 


1.9 


0.37 


0.81 




Baseline 


100.0 


0.0 


0.0 


0.00 


0.00 


+0.00 


0.01 


0.11 


Planning (e.g., MS Project) 


Spring 


100.0 


0.0 


0.0 


0.02 


0.14 


Other 


Baseline 


94.3 


2.9 


2.9 


0.23 


0.77 


+0.49 


0.10 


0.46 


Spring 


77.8 


9.3 


13.0 


0.72 


1.19 


Internet/Research Tools Used by Students 




Baseline 


57.1 


28.6 


14.3 


1.23 


1.17 


+0.80 


0.73 


1.12 


Internet Browser (e.g., Netscape) 


Spring 


33.4 


11.1 


55.5 


2.17 


1.31 




Baseline 


100.0 


0.0 


0.0 


0.09 


0.28 








CD Reference (encyclopedias, etc.) 


Spring 


98.2 


1.9 


0.0 


0.06 


0.30 


+0.00 


0.05 


0.29 


Communications 


Baseline 


100.0 


0.0 


0.0 


0.03 


0.17 


+0.45 


0.02 


0.25 


Spring 


94.3 


9.3 


0.0 


0.24 


0.61 










Baseline 


94.3 


2.9 


2.9 


0.23 


0.65 


+0.26 


0.09 


0.45 


Other 


Spring 


87.1 


3.7 


9.3 


0.41 


0.94 
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Student Computer Activities 




Percent Observed 


Florida EETT 


National Norm 


The extent to which each of the following was 


None 

or 




Frequently 

or 




Standard 






Standard 


Observed in the classroom. 




Rarely 


Occasionally 


Extensively 


Mean 


Deviation 


ES 


Mean 


Deviation 


Educational Software Used by Students 




















Drill/Practice/Tutorial 


Baseline 

Spring 


65.7 

79.6 


17.1 

13.0 


17.2 

7.5 


1.06 

0.76 


1.24 

1.01 


-0.27 


0.73 


1.06 




Baseline 


94.3 


5.7 


0.0 


0.14 


0.49 


-rO.33 


0.07 


0.31 


Problem Solving (e.g., SimCity) 


Spring 


90.7 


5.6 


3.7 


0.26 


0.73 


Process Tools (e.g., Geometer's 


Baseline 


94.3 


2.9 


2.9 


0.17 


0.75 


-rO.13 


0.03 


0.27 


Sketchpad) 


Spring 


87.1 


9.3 


3.7 


0.31 


0.80 


Other 


Baseline 


97.2 


2.9 


0.0 


0.09 


0.37 


+0.32 


0.21 


0.66 


Spring 


88.9 


5.6 


5.6 


0.30 


0.82 








Testing Software Used by Students 


Individualized/Tracked (e.g.. 


Baseline 


91.4 


5.7 


2.9 


0.26 


0.70 


+0.23 


0.52 


0.91 


Accelerated Reader) 


Spring 


83.3 


11.1 


5.6 


0.48 


0.97 


Generic 


Baseline 


100.0 


0.0 


0.0 


0.03 


0.17 


+0.26 


0.02 


0.20 




Spring 


98.2 


0.0 


1.9 


0.11 


0.46 










Baseline 


94.3 


5.7 


0.0 


0.14 


0.49 


-0.26 


0.08 


0.41 


Other 


Spring 


100.0 


0.0 


0.0 


0.04 


0.19 



Meaningfulness of Computer Activities* 



Low level use of computers 


Baseline 


68.6 


8.6 


22.8 


1.17 


1.32 


-0.36 


0.84 


1.16 


Spring 


79.6 


14.8 


5.6 


0.78 


0.90 


Somewhat meaningful use of 


Baseline 


77.1 


22.9 


0.0 


0.86 


0.77 


+0.42 


0.75 


1.00 


computers 


Spring 


48.1 


44.4 


7.5 


1.28 


1.05 


Meaningful use of computers 


Baseline 

Spring 


65.7 

40.7 


25.7 

20.4 


8.6 

38.9 


0.94 

1.87 


1.06 

1.26 


+0.83 


0.86 


1.21 


Very meaningful use of computers 


Baseline 


91.4 


5.7 


2.9 


0.31 


0.72 








Spring 


64.8 


13.0 


22.2 


1.11 


1.31 


+0.77 


0.39 


0.88 


Scale: 0 = Not Observed; 1 = Rarely; 2 


= Occasionally; 3 = Frequently; 4 = 


Extensivel 


/ 



Note, item percentages may not total 100% because of missing data. 

*Meaningfulness of Computer Activities Scale 

1. Low level use of computers; activities in general required no critical thinking, e.g., used computer applications for 
copying text or free-time drawing, or used educational software for drill & practice, tutorials, or games. 

2. Somewhat meaningful use of computers: activities in general required very little problem-solving or critical 
thinking and used computer applications or educational software in a limited manner. 

3. Somewhat meaningful use of computers: activities were problem-based, required some critical thinking skills, and 
some use of computer applications to locate and/or process information or some manipulation of educational 
software variables to reach solutions. 

4. Very meaningful use of computers: activities were based on meaningful problems, required critical thinking skills, 
and appropriate use of computer applications to locate and/or process information or manipulation of educational 
software variables to reach solutions. 



Subject Areas of Computer Activities 


Language 


Mathematics 


Science 


S. Studies 


Other 


Percent Not Observed 


Production Tools 


Baseline 


37.1 


17.1 


34.3 


31.4 


5.7 


22.9 


Spring 


74.1 


29.6 


44.4 


40.7 


9.3 


5.6 


Internet/Research 


Baseline 


25.7 


11.4 


31.4 


20.0 


2.9 


37.1 


Tools 


Spring 


59.3 


22.2 


44.4 


42.6 


3.7 


14.8 


Educational Software 


Baseline 

Spring 


37.1 

35.2 


20.0 

38.9 


8.6 

24.1 


5.7 

14.8 


2.9 

1.9 


54.3 

35.2 




Baseline 


20.0 


2.9 


2.9 


8.6 


0.0 


74.3 


Testing Software 


Spring 


29.6 


20.4 


16.7 


11.1 


0.0 


53.7 



Note. Item percentages may not total 100% because of missing data or activities involving more than one subject area. 
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SCU Multi-Class Inferential Statistics 

The Survey of Computer Use (SCU) observations are organized into 8 categories: 
“Computer Configuration”, “Computer Use”, “Frequency of Computer Type Use”, “Production Tools 
Used”, “Internet/Research Tools Used”, “Educational Software Used”, “Testing Software”, and 
“Overall Meaningful Use of Computers”. All rating categories with the exception of items under 
“Computer Configuration” and “Computer Use” are measured using a 5-point Likert scale (0=Not 
Observed, 1=Rarely, 2=Occasionally, 3=Frequently, and 4=Extensively). As a result, all SCU 
observations except “Computer Configuration” and “Computer Use” were analyzed using an 
adjusted alpha with Bonferroni correction, whereas the items under the first two categories were 
assessed using the normal alpha level (0.05). The SCU observation analyses outcomes are 
presented in Qs/wh and Qcsmh statistics in Table 5. The following SCU Multi-Class categories had 
significant findings: Computer Configuration, Digital Devices Available for Students, Student 
Computer Activities, and Overall Meaningful Use of Computers. Details of these differences are 
below. 

Computer Configuration. A larger number of computers or digital tools were observed 
during the spring observations as compared to the fall observations (Qsmh =Qcsmh=4.263, p=.039). 
Classroom computers observed during spring visits were also better equipped (more up-to-date) 
(Qsmh=Qcsmh=5.452, p=.020). Attention should be paid when interpreting the effect size associated 
with this item (i.e., negative as better) because of descending order of the rating scale (1 = up-to- 
date, 2 = Aging but adequate, 3 = Outdated/limited capacity, and 4 = No computers were 
observed). Spring observations found significantly fewer classrooms without students using 
computers {Qsmh =Qcsmh=1 0.028, p=.002). Once again, please note that the negative effect size is 
actually positive feedback on the spring over the fall results. 

Digitai Devices Avaiiabie for Student Use. There was significantly more Laptop 
computer usage during spring observations (Qsmh =Qcsmh= 14.612, p<.001). Use of “Digital 
Accessories” (Qsmh =Qcsmh= 9.131 , p=.0025) approached significance with an adjusted alpha of 
0.0017, with more usage during the spring as compared to the fall visits. 
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Student Computer Activities. Under “Internet/Research Tools Used by Students”, 
“Internet Browser” (Qsmh=Qcsmh=9. 192, p<.0024) also approached significance with an adjusted 
alpha of 0.001 7, meaning students usage of the Internet was more frequent during spring vs. fall 
observations. 

Overaii Meaningfui Use of Computers. When examining the meaningfulness of 
computer activities that were observed during the fall observations as compared to those seen 
during spring observations, two positive significant differences were revealed (see Figure 2). 
Specifically, “Meaningful use of computers” (Qsmh =Qcsmh=1 0.780, p=.001), and “Very meaningful 
use of computers” {Qsmh=Qcsmh=10.712, p=.001) were observed significantly more during the 
spring observations. 



iFall 
I Spring 





Meaningful Computer Use 
0.94 



Very Meaningful Comptuer Use 
0.31 
1.11 



Figure 2. Multi-Class SCU: Mean Scores of Significant Fall vs. Spring Differences in 
Meaningful use of Computers 
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TABLE 5 

SCU Multi-Class Means Comparison between Fall and Spring Using Mantel-Haentzel Test 



Item 


Qsmh 


P 


Qcsmh 


P 


Computer Configuration 










Classrooms most frequently had the following number of computers or digital tools (1 = 

None, 2= One, 3= 2-4, and 4 = No computers were observed) 


4.263* 


.039 


4.263* 


.039 


Classroom computers were most frequently (1 = up-to-date, 2 = Aging but adequate, 3= 
Outdated/limited capacity, 4 = 5-10, and 5 = 11 or more) 


5.452* 


.020 


5.452* 


.020 


In classrooms, computers were most frequently (1 = Connected to the internet, 2 = Not connected to 


0.500 


.480 


0.500 


.480 


the Internet, and 3 = No computers were observed) 

Total number of classrooms visited 


0.973 


.324 


0.973 


.324 


Total number of classrooms without students using computers 

Student Computer Use 


10.028* 


.002 


10.028* 


.002 


Classroom computers or digital tools were most frequently used by (1 = few, 2 = most, 3 = 

nearly all) 


2.168 


.141 


2.168 


.141 


Students most frequently worked with computers/digital tools (1 = alone, 2 = pairs, 3 = groups) 


0.312 


.576 


0.312 


.576 


Student computer literacy skills were most frequently (1 = poor, 2 = moderate, 3 = very good) 


0.444 


.505 


0.444 


.505 


Student keyboarding skills were most frequently (1 = poor, 2 = moderate, 3 = very good) 

Digital Devices used by Students 


0.108 


.743 


0.108 


.743 


Desktop computers. 


0.961 


.327 


0.961 


.327 


Laptop computers. 


14.612* 


<.001 


14.612* 


<.001 


Personal Data Assistants (PDA). 


1.333 


.248 


1.333 


.248 


Graphing calculators. 


3.800 


.051 


3.800 


.051 


Information Processors (e.g. Alphaboard). 


0.143 


.706 


0.143 


.706 


Digital Accessories (e.g. camera, scanner, probes). 

Production Toois Used by Students 


9.131 


.0025 


9.131 


.0025 


Word Processor 


2.379 


.123 


2.379 


.123 


Database 


1.455 


.228 


1.455 


.228 


Spreadsheet 


0.852 


.356 


0.852 


.356 


Draw/Paint/Graphics/Photo-imaging 


5.984 


.014 


5.984 


.014 


Presentation 


4.880 


.027 


4.880 


.027 


Authoring 


1.649 


.200 


1.649 


.200 


Concept Mapping 


1.563 


.211 


1.563 


.211 


Planning (e.g. MS Project) 


0.500 


.480 


0.500 


.480 


Other production tools 

Internet/Research Tools Used by Students 


5.410 


.020 


5.410 


.020 


Internet Browser 


9.192 


.0024 


9.192 


.0024 


CD Reference 


1.400 


.237 


1.400 


.237 


Communications 


5.079 


.024 


5.079 


.024 


Other Internet/Research Tools 


2.551 


.110 


2.551 


.110 


Educational Software Used by Students 










Drill/Practice/Tutorial 


6.203 


.013 


6.203 


.013 


Problem-Solving 


0.287 


.592 


0.287 


.592 


Process Tools 


0.681 


.409 


0.681 


.409 


Other educational software 


4.268 


.039 


4.268 


.039 


Testing Software Used by Students 










Individualized/Tracked 


1.130 


.288 


1.130 


.288 


Generic 


0.435 


.510 


0.435 


.510 


Other testing software 


2.380 


.123 


2.380 


.123 
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Table 5 continued 



Item 


Qsmh 


P 


Qcsmh 


P 


Overall Meaningful Use of Computers 

Low level use of computers 


4.047 


.044 


4.047 


.044 


Somewhat meaningful use of computers 


1.330 


.249 


1.330 


.249 


Meaningful use of computers 


10.780* 


.001 


10.780* 


.001 


Very meaningful use of computers 


10.712* 


.001 


10.712* 


.001 



*Statistically significant at aipha<.0017 



Targeted Classroom Observation Results 

Targeted observations were conducted in 38 classrooms in the fall and 76 classrooms in 
the late spring. The data were collected with SOMs and SCUs during prearranged one-hour 
sessions in which teachers were asked to implement a prepared lesson using the laptops. The 
targeted observation results are presented by data collection instrument. 

Targeted SOM 

The targeted SOM results revealed positive fall to spring increased usage of student- 
centered classroom practices (see Table 6). The most notable increase, as indicated by an Effect 
Size of -I-0.93, was in student engagement in “Project based learning” (Fall M = 0.58, Spring M = 

1 .97). other striking increases included more “Integration of subject areas” (ES = -t-0.87), 

“Teachers acting as coach/facilitators” (ES= -t-0.69), “Independent inquiry/research on the part of 
students” (ES= -t-0.58), “Cooperative/collaborative learning” (ES = -t-0.58), and “Sustained writing” 
(ES = -1-0.55). Additionally, the context for technology use shifted, with student use of “Technology 
as a learning tool/resource” increasing (Fall M = 1 .97, Spring M = 2.95, ES = -t-0.60) and 
“Technology as a delivery tool” decreasing (Fall M = 2.37, Spring M = 1 .58, ES = -0.48). 

Overall, “High academically focused class time” was observed frequently to extensively in 
approximately 70% of the classrooms during both the fall and spring semesters. However, there 
was an increase in the frequency with which a “High level of student attention, interest, and 
engagement” was seen in the spring as compared to the fall observations (Fall M = 2.66, Spring M 
= 3.12, ES= -rO.38). 
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TABLE 6 

Targeted School Observation Measure (SOM) Results 



Fall (Baseline) N = 38 

Spring N = 76 







Percent Observed 


Florida EETT 


National Norm 


The extent to which each of the following was 


None or 




Frequently 

or 




Standard 






Standard 


observed in the classroom. 




Rarely 


Occasionally 


Extensively 


Mean 


Deviation 


ES 


Mean 


Deviation 


Instructional Orientation 




















Direct instruction (lecture) 


Baseline 


31.6 


13.2 


55.2 


2.37 


1.46 


-0.28 


2.77 


1.01 


Spring 


42.1 


18.4 


39.4 


2.00 


1.40 




Baseline 


84.2 


2.6 


13.1 


0.58 


1.33 








Team teaching 


Spring 


79.0 


7.9 


13.1 


0.66 


1.20 


-hO.08 


0.81 


0.94 




Baseline 


68.4 


10.5 


21.1 


1.13 


1.47 


-rO.58 


1.08 


0.98 


Cooperative/collaborative learning 


Spring 


39.4 


6.6 


53.9 


1.97 


1.61 


Individual tutoring (teacher, peer, aide. 


Baseline 


86.8 


7.9 


5.2 


0.42 


0.95 


-rO.30 


0.77 


0.94 


adult volunteer) 


Spring 


77.7 


9.2 


13.2 


0.71 


1.09 


Classroom Organization 




















Ability groups 


Baseline 


86.8 


0.0 


13.1 


0.50 


1.31 


+0.15 


1.16 


1.33 


Spring 


80.2 


5.3 


14.5 


0.72 


1.44 




Baseline 


92.1 


2.6 


5.3 


0.26 


0.95 








Multi-age grouping 


Spring 


90.8 


5.3 


3.9 


0.26 


0.84 


0.00 


0.58 


1.03 




Baseline 


89.4 


0.0 


10.8 


0.42 


1.18 








Work centers (for individuals or groups) 


Spring 


86.8 


3.9 


9.2 


0.36 


0.93 


0.00 


1.35 


1.14 


Instructional Strategies 

Higher level instructional feedback (written 


Baseline 


60.5 


10.5 


28.9 


1.37 


1.46 


+0.40 


1.44 


1.15 


or verbal) to enhance student learning 


Spring 


34.2 


22.4 


43.4 


2.03 


1.45 


Integration of subject areas 


Baseline 


92.1 


5.3 


2.6 


0.24 


0.79 


+0.87 


0.54 


0.80 


(interdisciplinary/thematic units) 


Spring 


57.8 


7.9 


34.2 


1.33 


1.56 


Project-based learning 


Baseline 


84.2 


0.0 


15.8 


0.58 


1.31 










Spring 


40.8 


6.6 


52.6 


1.97 


1.70 


+0.93 


0.45 


0.76 


Use of higher-level questioning strategies 


Baseline 


65.8 


15.8 


18.5 


1.03 


1.33 










Spring 


51.3 


13.2 


35.6 


1.64 


1.49 


+0.43 


1.63 


1.12 


Teacher acting as a coach/facilitator 


Baseline 


36.9 


36.8 


26.3 


1.68 


1.36 










Spring 


19.7 


18.4 


61.8 


2.58 


1.24 


+0.69 


2.29 


1.18 


Parent/community involvement in learning 


Baseline 


94.7 


0.0 


5.3 


0.21 


0.91 


+0.11 


0.31 


0.60 


activities 


Spring 


90.7 


2.6 


6.5 


0.32 


0.91 


Student Activities 




















Independent seatwork (self-paced 


Baseline 


47.4 


18.4 


34.2 


1.71 


1.54 


-0.34 


2.41 


0.98 


worksheets, individual assignments) 


Spring 


63.1 


11.8 


25.0 


1.17 


1.40 


Experiential, hands-on learning 


Baseline 


52.6 


21.1 


26.3 


1.34 


1.49 










Spring 


48.7 


14.5 


36.8 


1.67 


1.71 


+0.25 


1.20 


1.00 


Systematic individual instruction 


Baseline 


89.4 


0.0 


10.5 


0.42 


1.18 










Spring 


85.5 


3.9 


10.5 


0.47 


1.10 


+0.09 


0.44 


0.76 


Sustained writing/composition (self- 


Baseline 


94.7 


5.3 


0.0 


0.13 


0.47 








selected or teacher-generated topics) 


Spring 


85.6 


9.2 


5.3 


0.49 


0.87 


+0.55 


0.75 


0.86 


Sustained reading 


Baseline 


94.7 


0.0 


5.2 


0.21 


0.81 








Spring 


84.2 


9.2 


6.5 


0.55 


1.00 


+0.44 


1.08 


0.98 
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Percent Observed 



National Norm 



Florida EETT 



The extent to which each of the following was 


None or 




Frequently 

or 




Standard 






Standard 


observed in the classroom. 




Rarely 


Occasionally Extensively 


Mean 


Deviation 


ES 


Mean 


Deviation 


Independent inquiry/research on the part 
of students 


Baseline 

Spring 


65.8 

36.8 


10.5 

11.8 


23.7 

51.3 


[TTr 

2.04 


1.64 

1.53 


+0.58 


0.32 


0.65 


Student discussion 


Baseline 

Spring 


76.3 

43.4 


2.6 

22.4 


21.0 

34.2 


0.97 

1.62 


1.42 

1.41 


-hO.43 


0.89 


1.11 


Technology Use 




















Computer for instructional delivery (e.g. 
CAI, drill & practice) 


Baseline 

Spring 


31.6 

51.3 


18.4 

13.2 


50.0 

35.5 


2.37 

1.58 


1.70 

1.59 


-0.48 


0.96 


0.97 


Technology as a learning tool or resource 


Baseline 


39.5 


21.1 


39.4 


1.97 


1.64 








(e.g. Internet research, spreadsheet or 
database creation) 


Spring 


15.7 


11.8 


72.4 


2.95 


1.37 


-hO.60 


0.80 


0.98 


Assessment 


Baseline 


97.4 


2.6 


0.0 


0.11 


0.39 








Performance assessment strategies 


Spring 


84.2 


11.8 


3.9 


0.38 


0.85 


-hO.47 


0.50 


0.83 


Student self-assessment (portfolios, 


Baseline 


97.4 


0.0 


2.6 


0.11 


0.65 


-hO.56 


0.32 


0.65 


individual record books) 


Spring 


80.2 


7.9 


11.8 


0.55 


1.10 


Summary Items 

High academically focused class time 


Baseline 


7.9 


21.1 


71.1 


3.08 


1.08 


0.00 


3.33 


0.80 




Spring 


3.9 


23.7 


72.3 


3.12 


0.91 


High level of student attention, interest, 


Baseline 


15.8 


21.1 


63.1 


2.66 


1.19 


-hO.38 


3.12 


0.83 


engagement 


Spring 


6.6 


17.1 


76.3 


3.12 


0.92 



Scale: 0 = Not Observed; 1 = Rarely; 2 = Occasionally; 3 = Frequently; 4 = Extensively 



SOM Targeted Inferential Statistics 

As shown in Table 7, a significant upward trend between observations was found on five 
SOM items: (1) “Integration of subject areas” (p<.001), (2) “Project-based learning” (p<.001), (3) 
“Teacher acting as a coach/facilitator” (p=.001), (4) “Independent inquiry/research on the part of 
students” (p<.001), and (5) “Technology as a learning tool or resource” (p=.001), meaning 
significantly more activities in these five areas were observed during spring observations 
compared to those conducted in the fall. 

As revealed by Qcsmh and its associated p-value, these five areas also had significantly 
higher mean responses at the spring observation (see Table 6 and Figure 3). In particular, the 
associated effect sizes (ranging from .58 to .93) are substantially large. An additional three SOM 
items approached significance with moderate to substantial effect sizes. They are 
“Cooperative/collaborative learning” (p=.005), “Higher-level instructional feedback to enhance 
student learning” (p=.002), and “Use of higher-level questioning strategies” (p=.005). 
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TABLE 7 

SOM Targeted Means Comparison between Fall and Spring Using Mantel-Haentzel Test 



Item 


Qsmh 


P 


Qcsmh 


P 


Instructional Orientation 










Direct instruction (lecture) 


0.414 


.520 


0.414 


.520 


Team teaching 


0.274 


.600 


0.274 


.600 


Cooperative/collaborative learning 


8.022 


.005 


8.022 


.005 


Individual tutoring (teacher, peer, aide, adult volunteer) 

Classroom Organization 


1.080 


.299 


1.080 


.299 


Ability groups 


0.058 


.810 


0.058 


.810 


Multi-age grouping 


0.107 


.744 


0.107 


.744 


Work centers (for individuals or groups) 

Instructional Strategies 


3.990 


.046 


3.990 


.046 


Higher-level instructional feedback (written or verbal) to enhance 
student learning 


9.186 


.002 


9.186 


.002 


Integration of subject areas (interdisciplinary/thematic units) 


16.076* 


<.001 


16.076* 


<.001 


Project-based learning 


16.848* 


<.001 


16.848* 


<.001 


Use of higher-level questioning strategies 


7.900 


.005 


7.900 


.005 


Teacher acting as a coach/facilitator 


10.608* 


.001 


10.608* 


.001 


Parent/community involvement in learning activities 

Student Activities 


0.178 


.673 


0.178 


.673 


Independent seatwork (self-paced worksheets, individual 
assignments) 


7.438 


.006 


7.438 


.006 


Experiential, hands-on learning 


1.331 


.249 


1.331 


.249 


Systematic individual instruction (differential assignments geared to 
individual needs) 


2.350 


.125 


2.350 


.125 


Sustained writing/composition (self-selected or teacher-generated 
topics) 


3.597 


.058 


3.597 


.058 


Sustained reading 


0.910 


.340 


0.910 


.340 


Independent inquiry/research on the part of students 


11.258* 


<.001 


11.258* 


<.001 


Student discussion 


6.498 


.011 


6.498 


.011 


Technology Use 










Computer for instructional delivery (e.g., CAI, drill & practice) 


3.763 


.052 


3.763 


.052 


Technology as a learning tool or resource (e.g., Internet research, 
spreadsheet creation, multi-media, CD Rom, Laser disk) 


10.320* 


.001 


10.320* 


.001 


Assessment 










Performance assessment strategies 


1.048 


.306 


1.048 


.306 


Student self-assessment (portfolios, individual record books) 

Summary Items 


2.667 


.102 


2.667 


.102 


High academically focused class time 


0.987 


.321 


0.987 


.321 


High level of student attention/interest/engagement 


6.290 


.012 


6.290 


.012 



'statistically significant at alpha<.0019 
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4 
3 
2 
1 
0 

■ Fall 

■ Spring 

Figures. Targeted SOM: Significant Fall vs. Spring Differences 

Targeted SCU 

Following the same trend as seen in the Multi-class results, the number of classrooms 
with “1 1 or more” computers available for student use increased from 64.9% in the fall to 87.1% in 
the spring, with nearly all (98.6%) considered as “Up-to-date” in the spring (see Table 8). There 
was also an increase (Fall = 62.2%; Spring = 71 .4%) in the percentage of classrooms in which the 
laptops were used by “nearly all” of the students, as well as an increase in the percentage of 
students rated with “very good” computer literacy skills (Fall = 18.9%; Spring = 55.7%) and 
keyboarding skills (Fall = 16.2%; Spring = 35.7%). The results revealed that during the fall and 
spring, students more frequently worked alone when using the laptop (Fall = 70.3%; Spring = 

71 .4%), however, there was a fall to spring increase in the frequency with which students worked 
in pairs during laptop use (Fall = 10.8%; Spring = 22.9%) 

Also following the same pattern of progress as the Multi-class results, the greatest fall to 
spring increase was in student use of the Internet, which was seen “Frequently” to “Extensively” in 
over one-half (55.7%) of the spring observations as compared to less than one-fourth (21 .6%) of 
the fall observations. This gain yielded an impressive Effect Size of -1-0.64. Other striking 
increases were in student use of “Presentation” software (Fall M = 0.43, Spring M = 1 .43, ES = 
-1-0.73) and “Other” types of “Production Tools” (Fall /W = 0.1 1 , Spring M = 0.81 , ES = -t-0.67). 
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Listed examples of these tools included Puzzle Maker, Photoshop and Note Taker. Spring 
observations revealed that student use of “Production Tools” and “Internet/Research Tools” were 
most often associated with Language Arts (Production Tools = 44.3%, Internet/Research Tools = 
35.7%) and Science (Production Tools = 38.6%, Internet/Research Tools = 31.4%). 

Although the targeted results are positive, it should be noted that although students were 
observed using 18 of the 20 computer applications listed on the SCU, the majority of the tools 
were used infrequently as seen in Mean scores for all but 3 of 1 8 uses that were at or below 1 .00, 
which equals “Rarely” observed. 

Meaningfulness of Computer Activities. Significant positive gains were seen from fall to 
spring regarding the meaningfulness of computer activities that were implemented during the 
targeted observations (see Table 8). Specifically, “Meaningful use of Computers” defined as 
“activities were problem-based, required some critical thinking skills, and some use of computer 
applications to locate and/or process information or some manipulation of educational software 
variables to reach solutions” increased from a Mean of 0.97 in the fall to 1 .94 in the spring (ES = 
-1-0.64). Encouragingly, the teachers demonstrated increased ability to implement lessons that 
engaged students in “Very meaningful use of computers” in which “activities were based on 
meaningful problems, required critical thinking skills, and appropriate use of computer applications 
to locate and/or process information or manipulation of educational software variables to reach 
solutions” (Fall M = 0.57, Spring M = 1 .27, ES = -t-0.49). Just as the “meaningfulness” of computer 
activities was observed to increase, the occurrence of “Low level use of computers” (activities in 
general required no critical thinking, e.g., used computer applications for copying text or free-time 
drawing, or used educational software for drill & practice, tutorials, or games) showed a fall to 
spring decrease (Fall M = 1.00, Spring M = 0.54, ES = -0.39). 



Florida’s EETT 2006-2007 Evaluation Report 24 




TABLE 8 



Targeted SCU Data Summary 

Fall (Baseline) N = 37 
Spring N = 70 



Computer Configuration 


Florida 

EETT 


Percent Observed 


Percentages of classrooms with the following numbers of computers or digital tools: 




None; One, or 2 -4 


Baseline 

Spring 




21.6 

2.9 


5-10 


Baseline 

Spring 




13.5 

10.0 


1 1 or more 


Baseline 




64.9 


Spring 




87.1 


Percentages of classrooms in which the majority 


of computers were: 






Up-to-date 


Baseline 

Spring 




81.1 

98.6 


Aging, but adequate 


Baseline 

Spring 




10.8 

1.4 


Outdated/limited capacity 


Baseline 

Spring 




2.7 

0.0 


Percentages of classrooms in which the majority 


of computers were: 






Connected to the Internet 


Baseline 

Spring 




83.8 

97.1 


Student Computer Use 


Percentage of classrooms in which computers or digital tools were used by: 






Few (less than 10%) to Some (about 10- 


Baseline 




16.2 


50%) students 


Spring 




10.0 


Most (about 51-90%) students 


Baseline 

Spring 




2.7 

10.0 


Nearly all (91-100%) students 


Baseline 

Spring 




62.2 

71.4 


Percentage of classrooms in which students worked with computers or digital tools: 




Alone 


Baseline 

Spring 




70.3 

71.4 


In pairs or small groups 


Baseline 




10.8 


Spring 




22.9 


Percentage of classrooms in which student computer literacy skills were: 






Poor 


Baseline 

Spring 




8.1 

0.0 




Baseline 




43.2 


Moderate 


Spring 




25.7 




Baseline 




18.9 


Very good 


Spring 




55.7 


Not observed 


Baseline 

Spring 




29.7 

18.6 


Percentage of classrooms in which student keyboarding skilis were: 






Poor 


Baseline 

Spring 




18.9 

1.4 




Baseline 




32.4 


Moderate 


Spring 




37.1 




Baseline 




16.2 


Very good 


Spring 




35.7 


Not observed 


Baseline 




32.4 


Spring 




25.7 
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Table 8 continued 



Digital Devices Used by Students 




Not or Rarely Observed 


Occasionally 


Frequently or 
Extensively 


Desktop Computers 


Baseline 

Spring 


100.0 

77.2 


0.0 

12.9 


0.0 

10.0 


Laptop Computers 


Baseline 

Spring 


27.0 

24.3 


10.8 

4.3 


62.2 

71.4 


Personal Data Assistants (PDA) 


Baseline 

Spring 


97.3 

94.3 


0.0 

2.9 


2.7 

2.9 


Graphing Calculator 


Baseline 

Spring 


100.0 

94.3 


0.0 

1.4 


0.0 

4.3 


Information Processor (e.g., Alphaboard) 


Baseline 

Spring 


100.0 

98.6 


0.0 

1.4 


0.0 

0.0 


Digital Accessories (e.g., camera, scanner, 


Baseline 


97.3 


0.0 


2.7 


probes) 


Spring 


77.1 


7.1 


15.7 



Note. Item percentages may not total 100% because of missing data. 



Student Computer Activities 




Percent Observed 


Florida EETT 


National Norm 


The extent to which each of the following was 


None 

or 




Frequently 

or 




Standard 






Standard 


Observed in the classroom. 




Rarely 


Occasionally 


Extensively 


Mean 


Deviation 


E$ 


Mean 


Deviation 


Production Tools Used by Students 




Baseline 


64.9 


8.1 


27.0 


1.22 


1.65 


0.00 


0.50 


0.90 


Word Processing 


Spring 


62.9 


7.1 


30.0 


1.17 


1.54 




Baseline 


97.3 


0.0 


2.7 


0.11 


0.52 


-0.28 


0.02 


0.19 


Database 


Spring 


100.0 


0.0 


0.0 


0.00 


0.00 




Baseline 


94.6 


0.0 


5.4 


0.16 


0.69 


-0.16 


0.07 


0.35 


Spreadsheet 


Spring 


95.7 


0.0 


2.9 


0.10 


0.51 




Baseline 


86.5 


8.1 


5.4 


0.38 


1.04 








Draw/Paint/Graphics 


Spring 


75.7 


10.0 


14.3 


0.74 


1.24 


+0.27 


0.19 


0.57 




Baseline 


86.5 


2.7 


10.8 


0.43 


1.14 








Presentation (e.g., MS PowerPoint) 


Spring 


54.3 


10.0 


35.7 


1.43 


1.57 


+0.73 


0.26 


0.68 




Baseline 


97.3 


0.0 


2.7 


0.11 


0.66 


+0.13 


0.02 


0.20 


Authoring (e.g., HyperStudio) 


Spring 


94.3 


0.0 


5.8 


0.23 


0.84 




Baseline 


91.9 


2.7 


5.4 


0.27 


0.87 


0.00 


0.08 


0.40 


Concept Mapping (e.g.. Inspiration) 


Spring 


92.9 


1.4 


5.7 


0.29 


0.92 


Planning (e.g., MS Project) 


Baseline 


100.0 


0.0 


0.0 


0.00 


0.00 


0.00 


0.01 


0.11 


Spring 


100.0 


0.0 


0.0 


0.00 


0.00 


Other 


Baseline 


97.3 


0.0 


2.7 


0.11 


0.52 


+0.67 


0.10 


0.46 


Spring 


74.3 


4.3 


21.4 


0.81 


1.41 


Internet/Research Tools Used by Students 




Baseline 


64.9 


13.5 


21.6 


1.19 


1.54 


+0.64 


0.73 


1.12 


Internet Browser (e.g., Netscape) 


Spring 


34.3 


10.0 


55.7 


2.20 


1.60 




Baseline 


100.0 


0.0 


0.0 


0.05 


0.23 








CD Reference (encyclopedias, etc.) 


Spring 


100.0 


0.0 


0.0 


0.00 


0.00 


-0.71 


0.05 


0.29 


Communications 


Baseline 


100.0 


0.0 


0.0 


0.00 


0.00 




0.02 


0.25 


Spring 


98.6 


0.0 


1.4 


0.07 


0.39 


+0.35 








Baseline 


86.5 


5.4 


8.1 


0.46 


1.02 




0.09 


0.45 


Other 


Spring 


88.5 


4.3 


7.1 


0.33 


0.91 


-0.21 






Educational Software Used by Students 


Drill/Practice/Tutorial 


Baseline 

Spring 


94.6 

90.0 


0.0 

2.9 


5.4 

7.1 


0.19 

0.31 


0.81 

0.84 


+0.13 


0.73 


1.06 
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Student Computer Activities 




Percent Observed 


Florida EETT 


National Norm 


The extent to which each of the following was 


None 

or 




Frequently 

or 




Standard 






Standard 


Observed in the classroom. 




Rarely 


Occasionally 


Extensively 


Mean 


Deviation 


E$ 


Mean 


Deviation 




Baseline 


97.3 


2.7 


0.0 


0.05 


0.33 


0.00 


0.07 


0.31 


Problem Solving (e.g., SimCity) 


Spring 


97.2 


0.0 


2.9 


0.11 


0.53 


Process Tools (e.g., Geometer's 
Sketchpad) 


Baseline 

Spring 


94.6 

90.0 


0.0 

1.4 


5.4 

8.6 


0.19 

0.30 


0.81 

0.87 


-hO.12 


0.03 


0.27 




Baseline 


89.2 


0.0 


10.8 


0.38 


1.11 


-0.21 


0.21 


0.66 


Other 


Spring 


94.3 


0.0 


5.8 


0.21 


0.83 



Testing Software Used by Students 



Individualized/Tracked (e.g.. 


Baseline 


97.3 


0.0 


2.7 


0.14 


0.67 


-rO.13 


0.52 


0.91 


Accelerated Reader) 


Spring 


91.4 


1.4 


7.1 


0.24 


0.81 


Generic 


Baseline 


100.0 


0.0 


0.0 


0.00 


0.00 


0.00 


0.02 


0.20 


Spring 


100.0 


0.0 


0.0 


0.00 


0.00 










Baseline 


97.3 


0.0 


2.7 


0.11 


0.66 


0.00 


0.08 


0.41 


Other 


Spring 


97.1 


0.0 


2.9 


0.09 


0.50 


Meaningfulness of Computer Activities* 


Low level use of computers 


Baseline 


73.0 


5.4 


21.6 


1.00 


1.56 


-0.39 


0.84 


1.16 


Spring 


85.7 


10.0 


4.3 


0.54 


0.90 


Somewhat meaningful use of 


Baseline 


67.6 


21.6 


10.8 


0.78 


1.13 


-rO.33 


0.75 


1.00 


computers 


Spring 


61.4 


21.4 


17.1 


1.16 


1.28 


Meaningful use of computers 


Baseline 

Spring 


73.0 

37.2 


2.7 

12.9 


24.3 

50.0 


0.97 

1.94 


1.44 

1.43 


-rO.64 


0.86 


1.21 


Very meaningful use of computers 


Baseline 


81.1 


5.4 


13.5 


0.57 


1.24 


-rO.49 


0.39 


0.88 


Spring 


65.7 


7.1 


27.1 


1.27 


1 .55 



Scale: 0 = Not Observed; 1 = Rarely; 2 = Occasionally; 3 = Frequently; 4 = Extensively 
Note. Item percentages may not total 100% because of missing data. 



Meaningfulness of Computer Activities Scale 

1. Low level use of computers: activities in general required no critical thinking, e.g., used computer applications for 
copying text or free-time drawing, or used educational software for drill & practice, tutorials, or games. 

2. Somewhat meaningful use of computers: activities in general required very little problem-solving or critical 
thinking and used computer applications or educational software in a limited manner. 

3. Somewhat meaningful use of computers: activities were problem-based, required some critical thinking skills, and 
some use of computer applications to locate and/or process information or some manipulation of educational 
software variables to reach solutions. 

4. Very meaningful use of computers: activities were based on meaningful problems, required critical thinking skills, 
and appropriate use of computer applications to locate and/or process information or manipulation of educational 
software variables to reach solutions. 



Subject Areas of Computer 












Percent Not 


Activities 




Language 


Mathematics 


Science 


S. Studies 


Other 


Observed 


Production Tools 


Baseline 


21.6 


8.1 


24.3 


10.8 


2.7 


35.1 


Spring 


44.3 


18.6 


38.6 


27.1 


11.4 


11.4 


Internet/Research 


Baseline 


8.1 


5.4 


18.9 


5.4 


2.7 


67.6 


Tools 


Spring 


35.7 


14.3 


31.4 


21.4 


8.6 


21.4 


Educational Software 


Baseline 

Spring 


8.1 

15.7 


5.4 

11.4 


5.4 

12.9 


0.0 

12.9 


2.7 

0.0 


81.1 

55.7 


Testing Software 


Baseline 

Spring 


5.4 

11. 


8.1 

2.9 


5.4 

10.0 


2.7 

11.4 


0.0 

0.0 


83.8 

68.6 



Note. Item percentages may not total 100% because of missing data or activities involving more than one subject area. 
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SCU Targeted Inferential Statistics 

The SCU Targeted observation analyses outcomes are presented in Qsmh and Qcsmh 
statistics in Table 9 and Figure 4. The following SCU categories had significant findings: 
Computer Configuration, Student Computer Activities, and Overall Meaningful Use of Computers. 
Details of these differences are below. 

Computer Configuration. As with the Multi-class observations, there were significantly 
more computers or digital tools observed in use during the spring observations as compared to the 
fall observations (Qsmh =Qcsmh= 1 0.1 1 9, p = .0015). Classroom computers observed during 
spring visits were newer and more up-to-date) (Qsmh=Qcsmh= 8.487, p = .004/ Again, attention 
should be paid when interpreting the effect size (Table 8) of this SCU item. As the ratings were 
arranged in descending order (1=up-to-date, 2=Aging but adequate, 3=Outdated/limited capacity, 
and 4=No computers were observed), the effect size should be interpreted in the opposite 
direction, with a negative sign indicating a more positive rating in the observation. Spring 
observations revealed more classroom computers connected to the Internet as compared to the 
fall (Qsmh =Qcsmh= 5.744, p= .01 7j. The ratings for this SCU item are: 1=connected to the 
Internet, 2=Not connected to the Internet, and 3=No computers were observed. Thus, as noted 
above, special attention should be made to interpret the effect size. 

Student Computer Activities. Using an adjusted alpha of 0.0017 to examine “Production 
Tools Used by Students”, the analysis revealed that students were found to have significantly 
increased use of “Other production tools” (Qsmh =Qcsmh= 14.781 , p < .001). As noted earlier, 
examples of these tools included Puzzle Maker, Photoshop, and Note Taker. 

Overaii Meaningfui Use of Computers. When examining the meaningfulness of 
computer activities that were observed during the fall Targeted observations as compared to those 
seen during spring observations, one significant difference was revealed (see Table 9). 

Specifically, using an adjusted alpha of 0.0017, students were found to have significantly 
increased “Meaningful use of computers” (Qsmh =Qcsmh=1 3.384, p<.001) during the spring 
observations. 
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TABLE 9 

SCU Targeted Means Comparison between Fall and Spring Using Mantel-Haentzel Test 



Item 


Qsmh 


P 


Qcsmh 


P 


Computer Configuration 










Classrooms most frequently had the following number of computers or digital tools (1 = 

None, 2= One, 3= 2-4, and 4 = No computers were observed) 


10.119* 


.0015 


10.119* 


.0015 


Classroom computers were most frequently (1 = up-to-date, 2= Aging but adequate, 3= 
Outdated/limited capacity, and 4 = No computers were observed) 


8.487* 


.004 


8.487* 


.004 


In classrooms, computers were most frequently (1 = Connected to the internet, 2 = Not connected to 


5.744* 


.017 


5.744* 


.017 


the Internet, and 3 = No computers were observed) 

Total number of classrooms visited 


1.990 


.158 


1.990 


.158 


Total number of classrooms without students using computers 

Student Computer Use 


0.248 


.619 


0.248 


.619 


Classroom computers or digital tools were most frequently by (1 = few, 2 = most, 3 = nearly aii) 


0.003 


.960 


0.003 


.960 


Students most frequently worked with computers/digital tools (i = alone, 2 = pairs, 3 = groups) 


0.192 


.662 


0.192 


.662 


Student computer literacy skills were most frequently (i = poor, 2 = moderate, 3 = very good) 


2.329 


.127 


2.329 


.127 


Student keyboarding skills were most frequently (i = poor, 2 = moderate, 3 = very good) 

Digital Devices used by Students 


3.084 


.079 


3.084 


.079 


Desktop computers. 


6.810 


.009 


6.810 


.009 


Laptop computers. 


0.063 


.802 


0.063 


.802 


Personal Data Assistants (PDA). 


0.615 


.433 


0.615 


.433 


Graphing calculators. 


3.046 


.081 


3.046 


.081 


Information Processors (e.g. Alphaboard). 


0.033 


.855 


0.033 


.855 


Digital Accessories (e.g. camera, scanner, probes). 

Production Toois Used by Students 


5.539 


.019 


5.539 


.019 


Word Processor 


1.081 


.299 


1.081 


.299 


Database 


1.899 


.168 


1.899 


.168 


Spreadsheet 


0.118 


.732 


0.118 


.732 


Draw/Paint/Graphics/Photo-imaging 


0.472 


.492 


0.472 


.492 


Presentation 


9.049 


.003 


9.049 


.003 


Authoring 


0.371 


.543 


0.371 


.543 


Concept Mapping 


0.028 


.867 


0.028 


.867 


Planning (e.g. MS Project) 


- 


- 


- 


- 


Other production tools 

Internet/Research Tools Used by Students 


14.781* 


<.001 


14.781* 


<.001 


Internet Browser 


5.486 


.019 


5.486 


.019 


CD Reference 


5.628 


.018 


5.628 


.018 


Communications 


0.400 


.527 


0.400 


.527 


Other Internet/Research Tools 


0.310 


.578 


0.310 


.578 


Educational Software Used by Students 










Drill/Practice/Tutorial 


0.048 


.827 


0.048 


.827 


Problem-Solving 


0.215 


.643 


0.215 


.643 


Process Tools 


0.015 


.902 


0.015 


.902 


Other educational software 


0.435 


.510 


0.435 


.510 


Testing Software Used by Students 










Individualized/Tracked 


0.022 


.881 


0.022 


.881 


Generic 


- 


- 


- 


- 


Other testing software 


0.030 


.862 


0.030 


.862 
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Table 9 continued 



Item 


Qsmh 


P 


Qcsmh 


P 


Overall Meaningful Use of Computers 

Low level use of computers 


7.450 


.006 


7.450 


.006 


Somewhat meaningful use of computers 


0.417 


.519 


0.417 


.519 


Meaningful use of computers 


13.384* 


<.001 


13.384* 


<.001 


Very meaningful use of computers 


6.571 


.010 


6.571 


.010 



*Statistically significant at aipha<.0017 
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Meaningful Computer Use 


Production Tools (Other) 


■ Fall 


0.97 


0.11 


■ Spring 


1.94 


0.81 



Figure 4. Target SCU: 



Mean Scores of Significant Fall vs. Spring Differences 
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SUMMARY 



The primary purpose of this study was to address the foliowing question: 

■ What changes in tool-based, student-centered teaching happen as a 
result of the infusion of technology and professional development? 

Both the SOM and SCU Multi-Class and Targeted observations revealed significant fall to 
spring increases in the use of student-centered practices. For the SOM, significant increases were 
found for both the Multi-Class and Targeted observations for student engagement in “Project- 
based learning”, “Independent inquiry/research on the part of students”, and student use of 
“Technology as a learning tool or resource.” The SCU results from both the Multi-Class and 
Targeted observations yielded significant increases in students’ overall use of newer and more up- 
to-date computers (laptops) and positive trends toward increased uses of production tools and 
Internet/research tools to support learning. A key finding that emerged from the results was the 
significant increase in the frequency with which teachers implemented meaningful computer 
activities that engaged students in higher-order thinking and problem solving through effective use 
of laptop-based technology tools. 

These first year results show promising trends in that the Florida EETT program seems to 
be serving as a catalyst for positive changes from traditional teaching environments to ones that 
are student-centered and engage learners in the meaningful use of computers to enhance 
learning. However, the data also reveal room for continued growth due to the modest frequency 
with which most of these changed practices occurred. An additional consideration when reviewing 
the evaluation results is the possible bias that may occur due to observer involvement in the 
Florida EETT program. 
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